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ABSTRACT 

A faculty’s commitment to continuous learning and improvement 
is a critical dimension in defining schools as high-performing learning 
communities. When planning an improvement effort, a school’s staff needs a 
conceptual framework that outlines the dimensions of school improvement. The 
AEL Continuous School Improvement Questionnaire (CSIQ) is a self-report 
inventory that asks professional staff to rate their school on six major 
dimensions: learning culture; connections among school, family, and community; 
shared leadership; shared goals for learning; purposeful student assessment; 
and effective teaching. Extensive pilot and field testing have shown the CSIQ 
to be a reliable and valid measure of faculty commitment to continuous 
learning and improvement. This paper reports normative CSIQ data from 3,821 
staff in 132 schools in Kentucky, Tennessee, Virginia, and West Virginia that 
took part in field testing. Within the sample was a subgroup of 11 schools 
identified as continuously improving or high-performing schools. These schools 
were labeled "Known” schools. Schools were categorized by grade level and by 
Johnson Code indicating rural/urban locale. CSIQ scores were highly reliable. 
School’s grade-level had a slight to modest effect, with elementary schools 
and those containing elementary grades having higher total and subscale 
scores. CSIQ scores had no relationship to extent of rurality. Educators in 
"Known" schools almost always scored higher than their counterparts in schools 
of the same type. (Contains 15 statistical data tables.) (SV) 
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Ever since education has been recognized as a profession, attempts to improve schools 
have had an ebb-and-flow history. School reform and exemplary school programs have been 
implemented with various degrees of success. Many federal and state funding programs have 
directly or indirectly aimed to improve schools. 

The staff of a school planning an improvement effort faces two major challenges: 

(1) “getting a handle on” how to begin and (2) keeping the improvement process going. To 
address the first challenge, a logical starting point is assessing the school on the “scale” of 
improvement. To do this, schools need a conceptual framework that outlines the dimensions of 
school improvement. The school’s instructional program and the elements that impact it — such 
as home and community — include many interrelated items, some wholly or partially under the 
control of the school and others over which the school exercises no control. A conceptual 
framework needs to be workable; it cannot consist of a long “laundry list” of items yet it must 
have enough specificity to provide direction for action. 

Although there may be ancillary goals, such as improved teacher morale, the ultimate 
goal of school improvement is improved student performance. This goal has received increased 
prominence due to recent emphasis on proficiency and performance testing. 

Background of Instrument 

AEL, in its role as a regional educational laboratory, has been committed to research on 
school improvement efforts since 1966. Among AEL’s projects was Quest (1996-2000), a 
network of school communities located in Kentucky, Tennessee, Virginia, and West Virginia. 
Quest schools were dedicated to building learning communities that support high levels of 
student and adult performance. The Quest Network for Quality Learning Communities 
emphasized six components, or dimensions, essential for successful student learning — 
dimensions that evolved into the conceptual framework that supports the AEL Continuous School 
Improvement Questionnaire (AEL CSIQ). During their collaboration with AEL, Quest schools 
contributed to the AEL CSIQ research base and participated in the pilot test (Howley-Rowe, 
2000a, 2000b, 2000c, 2000d). 

AEL’s basis for school improvement is embodied in its research-based framework. 

AEL’s Framework for Transforming Low-Performing Schools into High-Performing Learning 
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Communities, shown in Figure 1, underlies the AEL CSIQ and other AEL products and services. 
The circle representing improved student performance sits in the middle, indicating that all 
school improvement efforts ultimately are directed to this goal. The dimensions of the 
framework are shared leadership, effective teaching, school/family/community connections, 
purposeful student assessment, shared goals for learning, and learning culture. The arrows 
outside the circle indicate the dynamic nature of this framework: the dimensions are interactive 
and require continuous action on the part of the school community. 




Figure 1. AEL’s Framework for the AEL CSIQ 



Purpose of Study 

A faculty’s commitment to continuous learning and improvement is seen as a critical 
dimension in defining schools as high-performing learning communities. The AEL Continuous 
School Improvement Questionnaire (AEL CSIQ), with satisfactory reliability and validity from 
several pilot and field tests, has been available for a short time, yet has been administered to the 
faculty in 132 schools. The purpose of this study is to report normative AEL CSIQ data for 
those 132 schools. These normative data are presented by type (level) of school, locale type 
(Johnson) codes, and schools nominated to be high performing. 
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Instrument 

The AEL Continuous School Improvement Questionnaire (AEL CSIQ) is a self-report 
inventory that asks the professional staff — principal(s), teachers, counselors, librarians, media 
specialists, and teacher aides — to provide their perceptions of how the school rates on six major 
dimensions. That is, the performance of the school on the AEL CSIQ is a function of the 
combined perceptions of the professional staff and is most useful when staff provide thoughtful 
replies to each item. 

The AEL CSIQ includes 60 randomly ordered items that comprise six subscales of 10 
items each. The symbols (e.g., Si), names, and descriptions of the six subscales follow. 

Si: Learning Culture. This subscale reflects whether the culture of the school promotes 
learning by all — students, staff, and administration. It reflects the extent to which the 
school emphasizes learning rather than passive compliance, is a safe but exciting place to 
be, and encourages curiosity and exploration. In addition, it indicates the extent to which 
teachers have opportunities and encouragement to reflect on practice, work with others, 
and try new ways of teaching. 

S2: School/Family/Community Connections. This subscale assesses the degree to 
which parents and community members are involved in and feel part of the school. It 
reflects the degrees to which they are kept informed, meaningful partnerships exist, 
communication is open, and diverse points of view are honored and respected. 

S3: Shared Leadership. This subscale reflects the extent to which leadership is viewed 
as being shared. It assesses whether school administrators dominate decision making or 
if there are mechanisms for involving teachers, students, and parents. Opportunities for 
leadership development among the members of the school community are assessed, as 
are the degree to which information is shared and the extent to which school 
administrators listen to and solicit the input of others. 
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S 4 : Shared Goals for Learning. This subscale assesses the extent to which the school 
has clear, focused goals that are understood by all members of the school community. In 
addition, it reflects whether shared goals affect what is taught and how teachers teach, 
drive decisions about resources, focus on results for students, and are developed and 
“owned” by many rather than a few. 

S 5 : Purposeful Student Assessment. This subscale reflects the extent to which student 
assessment data are viewed as meaningful; are used by teachers to guide instructional 
decisions; and are communicated to and understood by the greater school community, 
including teachers, parents, students, and other members of the community. 

S6: Effective Teaching. This subscale ascertains the extent to which teacher practice is 
aligned with research on effective teaching. It assesses whether teachers actively engage 
students in a variety of learning tasks, pose questions that encourage reflection and 
higher-order thinking, expect students to think critically, and use teaching strategies 
designed to motivate students. 

Each item is scored on a 6-point rating scale. Response options range from 1, "Is not present," to 
6, "Is present to a high degree." The scores of the items within a subscale are summed for a total 
subscale raw score. A total AEL CSIQ raw score is the sum of the six subscale scores. 

The AEL CSIQ is a paper-and-pencil measure, printed to be compatible with optical 
scanning equipment. It is printed on both sides of one sheet of ll"xl7" white paper that is 
folded and lightly perforated in the middle to yield two 8-!/2"xll" sheets. It is printed in three 
colors and includes room for basic school identification information (school name, district name, 
state) and the date to be provided by the respondent. The instrument also includes directions for 
completing it and a place on the last page for a unique code to be provided by AEL for office use 
only. 

Extensive pilot and field testing (Meehan, Cowley, Wiersma, Orletsky, Sattes, & Walsh, 
2002) has shown the AEL CSIQ to be reliable and valid measure of a faculty’s commitment to 
continuous learning and improvement. The internal consistency reliability estimates (Cronbach 
alpha) were computed on the subscales and total scores of 3,821 educators in 132 schools in 
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AEL’s four-state region of Kentucky, Tennessee, Virginia, and West Virginia. All of the internal 
consistency reliabilties were in the .90s; specifically, Si =.90, S2 = .93, S3 = .96, S4 = .93, S5 = 
.93, S6 = .96, and total score = .98. 

As a measure of the stability reliability of the AEL CSIQ, approximately 300 educators, a 
subgroup of the group named about, completed the instrument twice with about a three-week 
interval between the two administrations. These educators were located in 20 schools across 
AEL’s four-state region, including seven elementary, three middle, six high, and four 
middle/high schools. The test-retest correlations ranged from .66 to .81; specifically, Si = .70, S2 
= .75, S3 = .81, S4 = .76, S5 = .70, S5 = .66, and total score = .80. 

The correlation between the total score on the AEL CSIQ and the School Climate 
Questionnaire (SCQ; Manning, Curtis, & McMillen, 1996) was computed as a measure of 
concurrent validity of the former instrument. The SCQ contains 10 rating-scale items and these 
items relate to elements of school improvement. The SCQ was administered concurrently with 
the AEL CSIQ to 174 educators in the testing phase. The internal consistency reliability for that 
administration of the SCQ was .97, which is high. The correlation between the total score on the 
AEL CSIQ and the SCQ was .75, which is satisfactory. 

Another measure of the construct validity of the AEL CSIQ was through factor analyses. 
Three factor analyses were computed, one each for elementary schools and high schools, and one 
for the two groups combined. The elementary group included scores for about 1,750 educators 
from 81 schools and the high school group included scores for about 680 educators from 17 
schools. The factor analyses were principal axis, Varimax rotation factorings. From the three 
exploratory factor analyses, 1 7 factors with eigenvalues greater than 1 .0 were extracted. The 
single exception was the sixth factor for the high school group, which had an eigenvalue of .924, 
which was retained because it was high and the factor fit the fifth subscale well. In the three 
factor analyses, there were occasional loadings greater than .30 for items from a subscale other 
than the one associated with a factor; however, these were relatively few and these loadings 
tended to be only slightly above .30. Thus, as reported in Meehan, Cowley, Wiersma, Orletsky, 
Sattes, & Walsh (2002), the trio of factor analyses on the AEL CSIQ were exceptionally “clear.” 




5 



11 



T 



Sample 

The AEL CSIQ was designed to be a general inventory that can be used with educators at 
any school level from kindergarten through 12th grade. The instrument, or more correctly, three 
variations of length of the original instrument, were tested by AEL staff three times — one pilot 
test and two field tests. The data for this study of normative characteristics of the AEL CSIQ 
were drawn from the two field tests. Therefore, to describe the sample of schools and educators 
in the normative database, it is important to describe each of the samples in the field tests. This 
is done in the following paragraphs. 

The first field test of the 72-item version of the AEL CSIQ was administered in the fall of 
2000 to educators, mostly teachers, in AEL’s four-state region. AEL project staff, using contacts 
and networks of colleagues developed over the years, recruited volunteers to help get the 
instrument distributed and completed in schools. In essence then, school staff were 
“volunteered” to complete the instrument. Since one of the purposes of the field tests was to 
determine differences by school level, respondents’ school information was sought. In the first 
field test, a total of 2,093 educators in 79 different schools participated. This same 72-iem 
version of the AEL CSIQ was administered to educators in 25 schools in Tennessee in the winter 
of 2000. Again, the school staff were “volunteered” by an AEL contact person to complete the 
instrument in exchange for school reports for use in a collaborative school improvement project. 
So, even though the timing of this administration of the instrument was several months after the 
first administration of the 72-item version, this was considered part of the first field test by virtue 
of the same number of items in the instrument. 

Then, to make the AEL CSIQ more convenient to the respondent, but still retain 
satisfactory internal consistency reliabilities for the six subscales and the total score, the decision 
was made to reduce its length to 60 items, 10 per subscale. Also, the decision was made to place 
all items in random order, rather than grouped by subscale name, as in the pilot test and the first 
field test. This 60-item version of the AEL CSIQ was administered to the full faculties of 75 
schools in Tennessee that were participating in a school improvement project with AEL staff. In 
effect, then, this second field test consisted of a convenience sample of educators. 

Within the total of 1 32 shcools in the field tests, there was a special subgroup of schools 
that were nominated by either AEL or Tennessee Department of Education staff as being high- 
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performing schools and professional learning communities. These schools were identified as 
belonging in one of two categories: 

1. Identified as continuously improving based on involvement with AEL in the four-year 
QUEST network of schools and other projects with AEL, and 

2. Identified as high-performing schools by staff in the Tennessee Department of Education 
based on School Report Card data high on student achievement and the Tennessee Value 
Added Assessment system. 

These schools, which are referred to hereafter as the “Known” schools, were viewed by the AEL 
research team as possessing positive characteristics relative to continuous improvement and 
student achievement. It is crucial to note that the “Known” schools were nominated to possess 
the characteristics of high-performing learning communities — there was no guarantee that they 
were, in fact, high-performing communities. Indeed, one of the purposes of this effort was to 
study the normative data from these “Known” schools. There were 1 1 Known schools for this 
normative study: five elementary, one middle, three high, and two middle/high schools. 

Data Analyses 

Various data analyses were conducted for this normative study of the AEL CSIQ. These 
analyses will be described in this subsection. 

The data analyses began by combining all the field-test AEL CSIQ files together. This 
was necessary because some of the field-test administrations were with the 72-item version of 
the instrument, while the last administration was with the 60-item version. So, the first step in 
combining the files was to eliminate the same two items from the 72-item version file as were 
dropped to create the 60-item version. The second step before combining the field-test files was 
to drop those schools with less than ten AEL CSIQ respondents. Recall, some school staffs 
volunteered to complete the instrument, but there was no incentive or reward for doing so. Thus, 
completion of the instrument within school staffs varied. In some schools, only a few educators 
returned completed copies, while in other schools there was nearly total staff completion rates. 
After the file transformations were completed, the resulting database for the analyses included 
3,821 cases in 132 schools. 

The next step in data analyses was to compute various statistics on the full 132 school 
database. First, the schools were grouped by type (level). The schools were in numerous 
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configurations from high schools only to schools with grades Pre-K through 12. The schools in 
the full group were categorized into six types. These six types and the number of schools in each 
were: elementary - 81, middle - 19, high school — 17, middle/high - 10, PreK-12 - 3, and 
vocational - 2. Second, descriptive statistics were computed for each school type and for the full 
group on their AEL CSIQ scores. These statistics included the frequencies, means, and standard 
deviations for each subscale and the total score. Third, the internal consistency reliability 
estimates (Cronbach alpha) were computed for the same six school groups by the subscales and 
total score. Fourth, to supplement the average scores on the subscales and total score by the six 
groups, the minimum and maximum scores were computed for display purposes. 

The third step in the normative study analyses was to cut the full group of schools by the 
type of locale in which they were located. The National Center for Education Statistics (2000) 
has developed and posted a type of locale classification for individual schools from most urban 
to most rural. Developed by Frank Johnson, these locale types often are called the “Johnson 
Codes.” At the time of this study, there were seven Johnson Codes in the system. Since then, an 
eighth code — a second rural code — has been added. Each of the 132 schools was given its 
Johnson Code. The frequencies and means on the AEL CSIQ scores for the schools in each 
Johnson Code were computed; however, the frequencies for the subscales were not displayed to 
save space and also, because they were similar to those of the total score. 

For the fourth major step in data analyses, Pearson product-moment correlation 
coefficients among the AEL CSIQ subscales were computed. Although the three factor analyses 
with orthogonal rotation generated six independent factors for each group, the research team felt 
that the subscales were positively related in the larger construct of continuous learning and 
improvement and they wanted to know just how much they were related. 

Finally, the fourth major step in data analyses was to break out the Known schools from 
the full set of 132, leaving 121 schools, which will be referred to hereafter as the “Remaining” 
schools. Recall, the Known schools were nominated to be high-performing learning 
communities, but this nomination was done prior to the staff in these schools completing the 
AEL CSIQ. Indeed, one of the purposes of this norming study was to compare the scores of the 
educators in the Known schools to those of the Remaining schools. These comparisons were 
made by computing the frequencies and means for the respondents in the Known and Remaining 
schools on the six subscales and total score for elementary, middle, high, and middle/high school 
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groups. There were no PreK-12 or vocational schools nominated to be in the Known group. 
Comparison with inferential statistics (i.e., t-tests) were not computed because the schools were 
not selected randomly. 
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FINDINGS 



Several types of normative data on the AEL CSIQ are presented in this section. First, 
normative data for the six different types (levels) of schools are presented, mostly in tables. 
Second, normative data by the schools’ type of locale (Johnson Codes) are given. Third, the 
interrelationships among the six subscales and total score are given. Fourth, normative data for 
the eleven Known schools nominated to be high-performing learning communities compared to 
the Remaining schools are provided, again mostly in table format. 

Normative Scores by Type of School 

Tables 1 through 7 contain the AEL CSIQ normative statistics for the six school types 
and the entire group of 132 schools. These tables contain the frequencies, means, standard 
deviations, and Cronbach alpha internal consistency reliabilities for the six subscales and the 
total score. The frequencies column shows the number of respondents who completed all the 
items in the subscales and, for the Total, the number who completed all 60 items. Respondents 
occasionally omitted an item or two on a subscale and this accounts for the varying frequencies. 

Normative data in Tables 1-7 stabilize as the number of scores from the schools and 
educators increases. This initial study of the norms for the AEL CSIQ provided substantial 
numbers for some school types, notably elementary, middle, high school, and, to a lesser extent, 
middle/high. The PreK-12 and vocational schools had very limited numbers but, due to their 
unique configurations, they were retained as separate groups. 

All schools had at least 10 respondents completing the AEL CSIQ. The average number 
of respondents per school by school type was computed but not displayed in Tables 1-7. The 
average number of respondents (rounded) per school by school type were as follows: 



Elementary 


- 25 


Middle 


- 30 


High 


- 45 


Middle/High 


- 28 


PreK-12 


- 24 


Vocational 


- 18 



These averages were expected; that is, high schools had, on average, the largest number of staff. 
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Table 1 



Frequencies (N), Means (M), Standard Deviations (SD) and Cronbach Alpha 

Reliability Estimates (Y a ) for AEL CSIQ Subscales and Total Score 
for Elementary School Group 



Subscale 


N 


M 


SD 


Y a 


s, 


2018 


49.2 


7.3 


.89 


s 2 


2013 


47.3 


8.7 


.93 


S 3 


1964 


47.6 


10.1 


.96 


s 4 


1945 


49.3 


7.9 


.93 


S 5 


1972 


48.8 


7.9 


.93 


s 6 


2005 


50.8 


7.6 


.96 


Total 


1758 


292.9 


41.0 


.98 



Si: Learning Culture 

S 2 : School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S&: Effective Teaching 
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Table 2 



Frequencies (N), Means (M), Standard Deviations (SD) and Cronbacb Alph 

Reliability Estimates (Y a ) for AEL CSIQ Subscales and Total Score 

for Middle School Group 



Subscale 


N 


M 


SD 


T a 


s, 


566 


46.0 


7.5 


.89 


s 2 


564 


43.7 


9.4 


.94 


S 3 


553 


45.4 


10.8 


.96 


S 4 


551 


44.7 


9.1 


.93 


s 5 


547 


44.0 


8.7 


.93 


s 6 


570 


47.6 


7.8 


.95 


Total 


502 


270.2 


43.7 


.98 



Si: Learning Culture 

S 2 : School/Family/Community Connections 

S3: Shared Leadership 

S 4 : Shared Goals for Learning 

S 5 : Purposeful Student Assessment 

S 6 : Effective Teaching 



Table 3 






Frequencies (N), Means (M), Standard Deviations (SD) and Cronbach Alpha 

Reliability Estimates (T a ) for AEL CSIQ Subscales and Total Score 

for High School Group 



Subscale 


N 


M 


SD 


Ta 


s, 


774 


46.8 


7.2 


.89 


s 2 


765 


44.4 


8.6 


.93 


S 3 


761 


45.6 


10.1 


.96 


s 4 


754 


45.4 


8.3 


.92 


s 5 


752 


44.7 


8.6 


.93 


s 6 


773 


47.8 


7.8 


.95 


Total 


683 


274.4 


42.4 


.98 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 
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Table 4 



Frequencies (N), Means (M), Standard Deviations (SD) and Cronbach Alpha 

Reliability Estimates (T a ) for AEL CSIQ Subscales and Total Score 
for Middle/High School Group 



Subscale 


N 


M 


SD 


T a 


Si 


277 


46.0 


7.8 


.90 


s 2 


277 


43.2 


10.1 


.94 


S 3 


276 


43.8 


11.7 


.96 


s 4 


268 


43.4 


9.2 


.93 


S 5 


267 


43.3 


9.2 


.93 


s 6 


275 


48.1 


8.0 


.95 


Total 


249 


266.9 


45.3 


.98 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 
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Table 5 



Frequencies (N), Means (M), Standard Deviations (SD) and Cronbach Alpha 

Reliability Estimates (T a ) for AEL CSIQ Subscales and Total Score 
for PreK-12 School Group 



Subscale 


N 


M 


SD 


Y a 


s, 


73 


47.6 


7.2 


.89 


s 2 


72 


45.4 


7.8 


.90 


S 3 


72 


47.3 


10.8 


.97 


s 4 


69 


46.6 


7.2 


.89 


S 5 


68 


47.0 


8.1 


.92 


s 6 


72 


48.9 


6.2 


.91 


Total 


65 


283.3 


33.8 


.96 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S&: Effective Teaching 




15 21 



Table 6 



Frequencies (N), Means (M), Standard Deviations (SD) and Cronbach Alpha 

Reliability Estimates (Tq) for AEL CSIQ Subscales and Total Score 
for Vocational School Group 



Subscale 


N 


M 


SD 


Ya 


s, 


36 


39.8 


8.5 


.86 


s 2 


38 


37.0 


9.7 


.89 


S 3 


36 


41.7 


12.1 


.95 


s 4 


36 


39.6 


10.7 


.96 


s 5 


33 


35.7 


11.9 


.96 


s 6 


39 


43.7 


9.8 


.94 


Total 


28 


239.2 


57.5 


.98 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S 4 : Shared Goals for Learning 

S5: Purposeful Student Assessment 

S&: Effective Teaching 
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Table 7 



Frequencies (N), Means (M), Standard Deviations (SD) and Cronbach Alpha 

Reliability Estimates (Y a ) for AEL CSIQ Subscales and Total Score 
Across All School Groups 



Subscale 


N 


M 


SD 


Y a 


s, 


3744 


47.9 


7.5 


.90 


s 2 


3729 


45.7 


9.1 


.93 


S 3 


3662 


46.5 


10.5 


.96 


s 4 


3623 


47.2 


8.6 


.93 


s 5 


3639 


46.7 


8.7 


.93 


s 6 


3734 


49.4 


7.8 


.96 


Total 


3285 


283.0 


43.6 


.98 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 
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Inspection of the AEL CSIQ descriptive statistics in Tables 1-7 show some interesting 
patterns. Most of the subscale means are in the 40s with the notable exceptions of one in the 50s 
(elementary group) and four in the 30s (all in the vocational group) out of the possible 60 points. 
Although the subscale means tend to be rather similar within a school group, they tend to vary 
across the school groups. For example, the mean scores for the elementary group are the highest, 
followed by the PreK-12 group, and then followed by the high school group (except for S 6 with 
the middle/high group). Then, the middle school group has higher means than the middle/high 
group (except for S 6 again). On all subscales, the vocational group has the lowest mean scores. 

Looking at the total scores in Tables 1-7, out of a possible 360 points, they ranged from a 
high of 292.9 to a low of 239.2. The high mean was for the elementary group and the low mean 
was for the vocational group. In between those extremes were scores of the PreK-12 (283.3), 
high school (274.4), middle school (270.2), and middle/high school (266.9) groups. 

Variability in the AEL CSIQ scores is displayed in the fourth column of each of the first 
seven tables. Standard deviations ranged from 6.2 (S 6 for PreK-12 group) to 12.1 (S 3 for 
vocational group). Generally, the two groups with elementary grades — elementary and PreK- 
12 — had the smallest standard deviations. The standard deviations for the subscales in the 
middle and middle/high school groups were similar in that each group had one or two in the 7s, 
one in the 8 s, 2 in the 9s, and one in the 10s. The vocational school group had the most 
variability in the subscale scores with one in the 8 s, 2 in the 9s, one in the 10s, one in the 11 s, 
and one in the 12s. With respect to the total score standard deviation, the PreK-12 had the 
smallest at 33.8 and the vocational school had the largest at 57.5. 

The internal consistency reliability estimates (Cronbach alpha) for the AEL CSIQ 
subscale and total scores in Tables 1-7 are very high. The Cronbach alphas (column five) ranged 
from .86 (Si for the vocational group) to .98 (for six of the seven total scores). Across the 
groups, the middle/high group had all seven reliability estimates in the .90s and the elementary, 
middle, and high school groups had alphas in the .90s on six of the seven scores. Interestingly, 
the same subscale (Si) yielded the most alpha reliabilities in the .80s, but it was .89 in each case 
except one, where it was . 86 . 

Tables 8 and 9 present the minimum and maximum school means for the AEL CSIQ 
subscales and total score for the normative groups. These data are displayed in Table 8 for 
elementary, middle, and high schools and in Table 9 for middle/high, PreK-12, and vocational 
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schools. These data are presented as additional type-of-school results on the instrument. The 
numbers of schools in these groups vary greatly, from 81 in the elementary group to only 2 in the 
vocational group. While the first seven tables show the means for the subscales and total score 
for each group, these two tables show the extremes for the means and provide ranges for those 
means. 

The ranges for the means in Tables 8 and 9 varied greatly across and within the school 
groups. Across all groups, the ranges of the subscale means were from a low of just 0.1 points 
(S6 for vocational group) to 27.6 points (S 3 for elementary group). Across the six groups in both 
tables, there were seven subscale score ranges over 20 points and ten under five points. The 
elementary, middle, and middle/high school groups each had at least two pairs of subscale scores 
with ranges over 20 points. On the other extreme, all six AEL CSIQ subscale mean ranges for 
the vocational group were less than 5 points and four ranges were less than a single point. For 
the PreK-12 group, four of the subscale mean ranges were less than 5 points. 
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Table 8 



Minimum and Maximum School Means for the AEL CSIQ Subscales and Total 
Score for Elementary, Middle, and High School Groups 



Elementary Middle High School 

Subscale Min Max Min Max Min Max 



S, 

5 2 

5 3 

5 4 

55 

s 6 

Total 



40.0 

36.9 

31.2 
40.4 

37.3 

40.6 
235.2 



56.2 

56.7 

58.8 

58.1 

57.6 
57.5 

340.0 



39.2 

35.0 

31.9 

33.1 

35.1 

41.4 
223.7 



55.3 

55.3 

53.9 

53.3 

53.4 

56.0 

324.6 



39.2 
37.7 

34.2 

40.2 

38.1 

42.1 

239.9 



51.1 

51.3 

52.9 

50.6 

49.0 

51.6 

305.0 



Si: Learning Culture 

S 2 : School/Family/Community Connections 

S3: Shared Leadership 

S 4 : Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 



Table 9 



Minimum and Maximum School Means for the AEL CSIQ Subscales and Total 
Score for Middle/High, PreK-12, and Vocational School Groups 



Middle/High PreK -12 Vocational 



Subscale 


Min 


Max 


Min 


Max 


Min 


Max 


s, 


40.9 


53.4 


46.4 


48.2 


39.7 


40.0 


s 2 


30.6 


52.9 


43.2 


50.0 


36.7 


37.1 


S3 


27.6 


51.4 


33.8 


52.3 


40.9 


43.2 


s 4 


36.8 


48.9 


45.4 


47.3 


39.5 


39.7 


S 5 


36.4 


49.2 


46.9 


47.4 


34.1 


38.2 


s 6 


44.1 


54.1 


47.8 


49.6 


43.6 


43.7 


Total 


216.9 


307.0 


268.4 


291.6 


239.1 


239.2 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 



Normative Scores by Johnson Codes 

Schools where educators completed the AEL CSIQ were identified according to locale 
type (Johnson Codes), a classification system of seven categories of the most urban locale to the 
most rural. The Johnson Codes comprise a category system in which the first category (1) is the 
most urban and the seventh (7) is the most rural (NCES, 2000). 

The AEL CSIQ means by Johnson Codes are given in Table 10. These means go across 
types of schools, and the frequency row (N) gives the number of respondents for the total scales. 
The frequencies for the subscales are not given but, for the most part, they were slightly greater 
than the frequencies for the total. 

An inspection of the means in Table 10 shows that (1) overall, the means of the subscales 
varied little across the Johnson Codes, and (2) there was no linear pattern relating the size of the 
means to the extent of urbanity-rurality. The total scores had more variability simply because the 
total scale contains 60 items, six times the number of items in the individual subscales. 

The means of the most rural classification (7) were positioned somewhat in the middle of 
the means on most subscales and the total. For no measures did it have either the greatest or 
smallest mean. The most urban classification (1) also had means positioned somewhat in the 
middle. For total score, it had the third highest mean. Locale Type Code 3 quite consistently 
had the greatest mean (the only exception being S 2 ), but this result likely was a function of the 
specific schools, rather than its position in the Johnson Codes. There is no evidence that scores 
on the AEL CSIQ subscales are related to the extent of urbanity-rurality of the school locale. 
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Table 10 



Frequencies (N) and AEL CSIQ Subscale and Total Score Means 
by Johnson Locale Codes Across All School Types 



Scale 


1 


2 


Johnson Locale Codes 
3 4 5 


6 


7 


s, 


50.4 


47.0 


51.4 


46.5 


46.9 


50.7 


48.4 


S2 


46.8 


44.2 


49.3 


46.2 


46.5 


49.4 


44.7 


S3 


46.6 


46.2 


49.6 


46.4 


44.2 


49.0 


46.0 


s 4 


49.0 


46.8 


49.5 


47.1 


45.9 


48.6 


46.3 


S5 


48.7 


45.6 


49.1 


46.5 


45.3 


48.4 


46.6 


s 6 


51.2 


48.7 


51.6 


49.0 


48.0 


51.2 


49.0 


Total 


293.2 


278.1 


297.9 


282.3 


274.5 


296.7 


280.6 


N 


370 


1106 


66 


869 


107 


244 


523 



Si: Learning Culture 

S 2 : School/Family/Community Connections 

S3: Shared Leadership 

S 4 : Shared Goals for Learning 

S 5 : Purposeful Student Assessment 

Sg: Effective Teaching 
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Relationships Among the AEL CSIQ Subscales 

Table 1 1 contains the correlation coefficients (Pearson product-moment) among the AEL 
CSIQ subscales and the total score. The 21 correlations range from a low of .42 to a high of .89; 
one in the AOs, two in the ,50s, eight in the .60s, five in the .70s, and five in the .80s. The total 
score tends to correlate highly with all the subscale scores, in part because the total score consists 
of the sum of the subscale scores. 

The following results were found in the patterns of the subscale correlations in Table 11. 

1. All correlations were positive, as expected. 

2 . Overall, S 3 , Shared Leadership, had the lowest correlations with the other scales. 
This may be because Shared Leadership is more administrative than instructional, 
which is how the other subscales would be viewed. 

3. The highest subscale correlation (.79) was between S 4 , Shared Goals for Learning, 
and S 5 , Purposeful Student Assessment. Possibly as goals for learning are 
established, appropriate assessment is seen as a necessary corollary. 

4. The lowest correlation (.42) was found between S 3 , Shared Leadership, and S 6 , 
Effective Teaching. Again, this low correlation may be because S 3 is viewed as 
administrative and S 6 is viewed as instructional. 

5. In terms of shared variance among subscale scores, percentages range from about 
18% to 62% 
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Table 11 



Correlation Coefficients Among AEL CSIQ Subscales and Total Score 



Subscale 


s 2 


S 3 


s 4 


s 5 


s 6 


Total 


s, 


.68 


.52 


.63 


.68 


.72 


.83 


s 2 




.61 


.66 


.70 


.62 


.85 


S3 






.65 


.58 


.42 


.78 


s 4 








.79 


.62 


.87 


S5 










.71 


.89 


s 6 












.80 



S\: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

Sg: Effective Teaching 
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Normative Scores for Known and Remaining Schools 

Within the full group of 132 schools, 1 1 schools were nominated as being high- 
performing learning communities because they appeared to possess positive characteristics 
relative to both student performance and a commitment to continuous learning and improvement 
on the part of the professional staff of the school. There were 1 1 Known schools: five 
elementary, one middle, three high, and two middle/high schools. Four of the school types had 
one or more schools in the Known group. However, the numbers of AEL CSIQ respondents in 
the Known schools were limited, even for the elementary and high school groups. The high 
school group had the largest number of respondents in the Known schools (about 155). 

Tables 12 through 15 present the normative data on the AEL CSIQ for respondents in the 
four types of Known schools and for the Remaining schools. Shown are the frequencies and 
means for the six subscales and total score by respondents in the Known and Remaining schools. 

The Known elementary school respondents (Table 12) consistently had higher AEL CSIQ 
mean scores than their counterparts in the Remaining schools. The mean for the Known total 
was more than 27 points greater than the total mean of the Remaining schools. 

The pattern was very different for the middle school group (Table 13), which consisted of 
AEL CSIQ scores from a single school). Except for the means for Si and S6, the means for 
respondents in the Known school were substantially lower than those in the Remaining schools. 
There were slightly more than 30 respondents for this Known school, so this does not comprise a 
large group for comparison. The scores of respondents in this Known school may have been 
more a function of the specific school than the fact that it was a middle school. 

The high school group consistently had greater means for the Known schools than those of 
the Remaining schools (Table 14). The difference in the total score means was slightly more 
than 15, with the Known school respondents being greater. 

The pattern for the middle/high school group was similar (Table 15). That is, respondents 
in the Known schools consistently had greater means than their counterparts in the Remaining 
schools. The difference between the total score means was almost 27 points. Again, the 
numbers of respondents in the Known schools were limited, those numbers being in the 50s. 

Comparisons to the Known schools must be done with caution, largely due to the limited 
numbers of AEL CSIQ respondents in these schools. Discounting the middle school group, 
which had respondents in only one Known school, the patterns showed that respondents in the 
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Known schools had greater means than those in the Remaining schools. This result in part 
supports the assumption that educators in nominated continuously improving and high- 
performing schools will score higher on the AEL CSIQ than those in schools not so nominated. 
Also, those patterns support the validity of the AEL CSIQ in measuring factors that impact the 
staffs progress toward continuous learning and improvement. 



Table 12 



Frequencies (N) and Means (M) for AEL CSIQ Subscales 
and Total by Known and Remaining Schools for 
Elementary School Group 



Known Remaining 



Subscale 


N 


M 


N 


M 


s, 


134 


54.0 


1884 


48.9 


s 2 


135 


53.6 


1878 


46.8 


S3 


125 


50.7 


1839 


47.4 


S 4 


124 


53.3 


1821 


49.0 


S5 


130 


53.2 


1842 


48.4 


s 6 


133 


54.1 


1872 


50.6 


Total 


110 


318.6 


1648 


291.2 



S i : Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 
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Table 13 



Frequencies (N) and Means (M) for AEL CSIQ Subscales 
and Total by Known and Remaining Schools for 
Middle School Group 



Subscale 


N 


Known 

M 


Remaining 
N M 


s, 


34 


47.3 


532 


46.0 


s 2 


34 


40.9 


530 


43.9 


S 3 


29 


31.9 


524 


46.1 


s 4 


30 


38.2 


521 


45.1 


s 5 


31 


39.2 


516 


44.3 


s 6 


34 


47.8 


536 


47.6 


Total 


27 


246.6 


475 


271.6 



Si: Learning Culture 

S2: School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S5: Purposeful Student Assessment 

S6: Effective Teaching 
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Table 14 



Frequencies (N) and Means (M) for AEL CSIQ Subscales 
and Total by Known and Remaining Schools for 
High School Group 



Known Remaining 



Subscale 


N 


M 


N 


M 


Si 


158 


49.4 


616 


46.1 


s 2 


159 


47.6 


606 


43.5 


S 3 


154 


47.2 


607 


45.2 


s 4 


156 


46.6 


598 


45.2 


S 5 


155 


47.2 


597 


44.1 


s 6 


155 


49.6 


618 


47.3 


Total 


142 


286.5 


541 


271.3 



Si : Learning Culture 

S 2 : School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S 5 : Purposeful Student Assessment 

S6: Effective Teaching 
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Table 15 



Frequencies (N) and Means (M) for AEL CSIQ Subscales 
and Total by Known and Remaining Schools for 
Middle/High School Group 



Subscale 


N 


Known 

M 


Remaining 
N M 


s, 


57 


51.3 


220 


44.6 


s 2 


57 


47.7 


220 


42.0 


S 3 


55 


44.8 


221 


43.6 


s 4 


56 


46.5 


212 


42.6 


S5 


54 


47.1 


213 


42.4 


s 6 


56 


52.9 


219 


46.9 


Total 


51 


288.2 


198 


261.4 



Si : Learning Culture 

S 2 : School/Family/Community Connections 

S3: Shared Leadership 

S4: Shared Goals for Learning 

S 5 : Purposeful Student Assessment 

S6‘- Effective Teaching 
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CONCLUSIONS AND RECOMMENDATIONS 



This study presents normative data on the AEL CSIQ from educators in 132 K- 12 
schools. This section presents conclusions and recommendations drawn from the normative 
data. 

Conclusions 

The scores on the 60-item AEL CSIQ from the administrations in this study are highly 
reliable, including the six individual subscales and the total score. 

The mean scores on the AEL CSIQ subscales are highly consistent and positioned about 
70% of the distance from the minimum to maximum scores. The AEL CSIQ provides adequate 
measurement of the constructs they measure with good location on the scale of measurement. 

The relatively high scores may be a function of the self-report nature of the and the fact that none 
of the 60 items was reverse scored. 

Type of school, that is level of school (elementary, high school), appears to have a slight 
to modest effect on the AEL CSIQ subscale and total score performance. Respondents in 
elementary schools and schools with elementary grades (PreK-12) had higher scores on the 
subscales and Total. 

There is no evidence that the scores on the AEL CSIQ are related to the extent of rurality- 
urbanicity of the school locale. Overall, there is no definite pattern between the extent of rurality 
and performance. Also, there was a limited number of scores in the most urban code, and this 
may be more a reflection of the specific school or schools in this normative study than the extent 
of urbanicity. 

Educators in schools nominated to be high-performing learning communities on the basis 
of their commitment to continuous learning and improvement almost always scored higher on 
the AEL CSIQ subscales and total score than their counterparts in Remaining schools of the 
same type. The most notable exception to that pattern was for the middle schools where the 
Remaining schools’ scores were higher on four of the six subscales and the total score. 

However, there was only one Known middle school, so those scores may have been a function of 
the specific school than the fact that it was a middle school. 
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The patterns of scores showing the respondents in the nominated Known schools having 
greater AEL CSIQ subscale and total score means (except the middle school group, as noted 
above) than of respondents in the Remaining schools supports the AEL assumption that a 
faculty’s commitment to continuous learning and improvement is the critical dimension in 
defining schools as high-performing learning communities. 

The implication of the positive correlations among the AEL CSIQ subscale is that 
perceived learning and improvement in one area tends to coincide with learning and 
improvement in other areas. Conversely, a decrease in one area of learning and improvement 
would tend to go with reduced scores in the other areas. These correlation coefficients reflect the 
holistic nature of continuous school learning and improvement. Although certain areas may be 
stronger or weaker than others, continuous school learning and improvement tends to move 
forward (or decline) in a unified manner rather than as segmented parts. 

Recommendations 

The AEL CSIQ should continue to be administered to schools and the results of these 
administrations should be added to the norming database. It is recommended that only whole 
school faculty complete the instrument as opposed to the minimum of ten, as used in this study. 
Too, efforts should be made to seek schools of all types to add to the database; especially needed 
are more middle, PreK-12, and vocational schools. Further, schools in other regions of the 
United States, other than AEL’s four states, should be recruited to complete the AEL-CSIQ. 

Even as the above recommendation is being implemented, AEL researchers should make 
concerted efforts to seek nominations for schools to be added to those in the Known group. 
Schools thought to be high-performing learning communities should be solicited to have their 
professional staff complete the AEL CSIQ. Consistently high scores on the subscales of the 
AEL CSIQ will help to assess the extent to which the faculty is committed to continuous learning 
and improvement, as measured by AEL’s instrument. 

With a database of 132 schools having completed the AEL CSIQ, norms should be 
developed for the six subscales and, perhaps, the total score. These norms should be developed 
for two major groups of schools — Known and Remaining — by school types. Also, in preparing 
the AEL CSIQ norms for the Known group schools, AEL should take a hard look at the scores 
for the 1 1 Known schools in this study. Certainly, the lone Known middle school should not 
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qualify as a high-performing school based on its rather low scores on the AEL CSIQ subscales. 
Perhaps other nominated Known schools in this study do not provide enough evidence that their 
faculty is committed to continuous learning and improvement. 

The value of computing and reporting the total AEL CSIQ score should be discussed by 
the AEL research team. It may not be useful to report to schools themselves, or anybody for that 
matter, on the total score for the instrument. Attending to school improvement efforts in any of 
the six AEL CSIQ constructs would be a major initiative itself and looking at the aggregation of 
all six of them in the total score would be daunting, at the least, and possibly overwhelming to 
practitioners. 

Schools involved in this effort received several different types of AEL CSIQ feedback, 
depending on several factors such as contact person requests, promises in exchange for 
participation, and being involved in an AEL initiative. What is needed now is a uniform school 
report format for their AEL CSIQ results. AEL staff should develop several school report 
formats and provide them to school staff. Then, AEL staff should solicit practitioners’ feedback 
regarding these AEL CSIQ school reports. A uniform format for the AEL CSIQ school report 
should be selected based on this feedback. 

Last, AEL staff should prepare resources to support the use of the AEL CSIQ in the field. 
An administration or users manual would be convenient to those interested in implementing the 
AEL CSIQ. Also a technical report on the development, testing, and refinement of the AEL 
CSIQ could help present and preserve the history and the psychometric properties of the 
instrument. These two resources — users manual and technical report — might be combined into 
one document. A review of the literature for each of the six constructs in the AEL CSIQ should 
be included in one of the support resources named here. 
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